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SYSTEM AND METHOD FOR IMPORTING PERFORMANCE DATA 
INTO A PERFORMANCE EVALUATION SYSTEM 

RELATED APPLICATIONS 

This application claims the benefit of U.S. 
Provisional Application Serial No. 60/084,794, entitled 
"METHOD AND SYSTEM FOR DETERMINING THE PERFORMANCE OF A 
CALL CENTER AGENT, " (Attorney's Docket 019143.0137), filed 
May 8, 1998, which is hereby incorporated by reference. 

This application is related to copending U.S. Patent 

Application Serial No. , entitled "SYSTEM AND 

METHOD FOR DEFINING THE ORGANIZATIONAL STRUCTURE OF AN 
ENTERPRISE IN A PERFORMANCE EVALUATION SYSTEM" (Attorney's 
Docket 019143.0269); copending U.S. Patent Application 

Serial No. , entitled "SYSTEM AND METHOD FOR 

GENERATING AN EVALUATION IN A PERFORMANCE EVALUATION 
SYSTEM" (Attorney's Docket 019143.0270); copending U.S. 

Patent Application Serial No. , entitled "SYSTEM 

AND METHOD FOR PROVIDING ACCESS PRIVILEGES FOR USERS IN A 
PERFORMANCE EVALUATION SYSTEM" (Attorney's Docket 
019143.0271); and copending U.S. Patent Application Serial 

No. , entitled "SYSTEM AND METHOD FOR GENERATING 

RESULTS IN A PERFORMANCE EVALUATION SYSTEM" (Attorney's 
Docket 019143.0273) . 

TECHNICAL FIELD OF THE INVENTION 

This invention relates generally to evaluation tools, 
and more particularly to a system and method for importing 
performance data into a performance evaluation system. 
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BACKGROUND OF THE INVENTION 

Evaluation tools are used to gather and record 
productivity and quality statistics for a business or other 
organization. Analysis of such information allows 

5 inefficiencies and other problems of the business to be 

identified and corrected. In this way, business 

performance is maximized. 

Traditionally, evaluation tools are implemented on 
stand-alone systems that are unable to effectively 

10 communicate with other resources in a network environment. 

Other problems include the use of predefined organizational 
structures that cannot be modified to match the 
organizational structure of a business. In addition, 
privileges, evaluations, and reports are generally hard- 

15 coded for each evaluation tool. Such customization leads 

to high implementation and administration cost. 
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SUMMARY OF THE INVENTION 

In accordance with the present invention, a 
performance evaluation system is provided that 
substantially eliminates or reduces disadvantages or 
5 problems associated with the previously developed systems. 

In particular, the present invention provides a performance 
evaluation system in which users may define configurations 
for importing performance data into the performance 
evaluation system. 

10 In one embodiment of the present invention, a data 

file is imported into the performance evaluation system by 
storing a plurality of user-defined data elements. A user- 
defined configuration is stored for the data file. The 
configuration is operable to identify a data item in the 

15 data file and to map the data item to a data element in the 

performance evaluation system. 

More specifically, in accordance with one embodiment 
of the present invention, the configuration associates the 
data item with a member based on a member name or code. 

20 The configuration may also identify a name, type and format 

of the data item. The data item may be identified based on 
its position in a delimited file. 

Technical advantages of the present invention include 
providing an improved performance evaluation system. In 

25 particular, the performance evaluation system allows users 

to define configurations for importing external 
productivity data into the performance evaluation system. 
As a result, data can be imported from a wide variety of 
external devices. In this way, the evaluation process is 

30 streamlined by having data in a centralized location for 

data analysis. 

Other technical advantages will be readily apparent to 
one skilled in the art from the following figures, 
description, and claims. 
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BRIEF DESCRIPTION OF THE DRAWINGS 

For a more complete understanding of the present 
invention and its advantages, reference is now made to the 
following description taken in conjunction with the 
5 accompanying drawings in which: 

FIGURE 1 is a block diagram illustrating a performance 
evaluation system in accordance with one embodiment of the 
present invention; 

FIGURE 2 is a block diagram illustrating an example 
10 call center using the performance evaluation system of 

FIGURE 1; 

FIGURE 3 illustrates details of the organization 
tables of FIGURE 1 in accordance with one embodiment of the 
present invention; 
15 FIGURE 4 illustrates details of the privilege tables 

of FIGURE 1 in accordance with one embodiment of the 
present invention; 

FIGURE 5 illustrates details of the plan tables of 
FIGURE 1 in accordance with one embodiment of the present 
2 0 invention; 

FIGURE 6 illustrates details of the productivity 
tables of FIGURE 1 in accordance with one embodiment of the 
present invention; 

FIGURE 7 illustrates details of the data import tables 
25 of FIGURE 1 in accordance with one embodiment of the 

present invention; 

FIGURE 8 illustrates details of an evaluation file 
stored in the evaluation tables of FIGURE 1 in accordance 
with one embodiment of the present invention; and 
30 FIGURE 9 illustrates details of the filter tables of 

FIGURE 1 in accordance with one embodiment of the present 
invention. 
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DETAILED DESCRIPTION OF THE INVENTION 

FIGURE 1 illustrates a performance evaluation system 
10 in accordance with one embodiment of the present 
invention. As described in more detail below, the 

5 performance evaluation system 10 uses productivity and 

quality data to evaluate the performance of an individual, 
group, process or other suitable type of item or operation. 

Referring to FIGURE 1, the performance evaluation 
system 10 is deployed on a three-tiered architecture. The 

5 three-tiered architecture includes client space 12, server 

application space 14 and database space 16. The client 
space 12 is implemented on a client platform 18 such as a 
work station, personal computer or other device capable of 
communicating between a user and a server. The server 

5 application and database spaces 14 and 16 are implemented 

on a server platform 20 such as a personal computer or 
other device capable of being remotely accessed over a 
network. 

The client and server platforms 18 and 20 are 

0 connected by a network 22. The network 22 may the Internet 

or other wide area network (WAN) , an enterprise intranet or 
other a local area network (LAN) , or other suitable type of 
link capable of communicating data between the client and 
server platforms 18 and 20. 

5 The client space 12 includes a web-browser 30 having 

a graphical user interface (GUI) 32. The graphical user 
interface 32 displays web pages 34 downloaded over the 
network 22 from the server application space 14. The web 
pages 34 may use HTML or Active X controls 36. The Active 

0 X controls 34 provide a rich interface 32 with superior 

scripting control. 

The server application space 14 includes a server 
engine 40, business objects 42, reports 44, charts 46, and 
a data importer 48. The server engine 42 includes active 

5 server pages (ASP) 50 that include server-side components 
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and scripting. The server-side components are specialized 
Active X components executed prior to delivery of web pages 
to the client space 12. The server-side scripting accesses 
the interfaces of the server-side components. Use of the 
active server pages 50 allows the web pages to be updated 
in response to changing sysbem and database settings. 

The business objects 42 call stored procedures in the 
database space 16 and perform preprocessing and expressions 
processing functions. In a particular embodiment, the 
business objects 42 include an organization object, a plan 
object, an evaluation object, an import service object, and 
a results object. In this embodiment, the organization 
object provides an interface for basic organizational 
functionality. The organization object also provides an 
interface to a class of services table giving the client 
space 12 access to allowed functions. The plan object 
provides an interface for the manipulation and use of 
evaluation guidelines. The evaluation object provides 
interfaces for performing evaluations. The import service 
object schedules and retrieves external data and places it 
into a delimited file. The import service object also 
provides a basic interface to the configuration settings. 
The results object receives information from the client 
space 12 for selecting reports based on the requested 
filters. Each of the objects may comprise one or more 
obj ects . 

The reports 44 may include both detail and summary 
reports. The detail reports provide information for 
members and elements of each member. In a particular 
0 embodiment, the detail reports include a productivity and 

quality report, a quality evaluation report, a productivity 
analysis report, a quality question report, a productivity 
element report, and a detail evaluation report. The 
productivity and quality report presents actual scores, 
5 maximum scores, and percentages of maximum for both quality 
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and productivity by an evaluation date for each member with 
group averages for all hierarchical levels. The quality 
evaluation report presents notes, actual scores, maximum 
score, and percentages of maximum for each question of an 
5 evaluation with subtotals by performance areas, evaluation, 

member, and any additional hierarchical levels. The 
productivity analysis report presents actual scores, 
maximum score, and percentage of maximum for each equation 
of productivity analysis with subtotals by member and by 

10 any additional hierarchical levels. The quality and 

question report presents actual scores for each question 
along with the total actual score, maximum score, and 
percentage of maximum for each performance area with 
subtotals by evaluation, member, and any additional 

15 hierarchical levels. The productivity element report 

presents actual data along with the equation-produced 
actual score, maximum score, and percentage of maximum for 
each element of productivity data with subtotals by member 
and any additional hierarchical levels. The detail 

20 evaluation report presents evaluation date, time, and 

status of all evaluations for a member subtotal by status, 
date, member, and evaluator. 

The summary reports provide average scores for a 
particular member or level. In a particular embodiment, 

25 the summary reports include a productivity and quality 

report, a quality evaluation report, a productivity 
analysis report, a quality question report and a 
productivity and quality comparison report. The 
productivity and quality report presents average scores for 

30 both quality and productivity for each member with group 

averages for all hierarchical levels. The quality 
evaluation report presents average scores by performance 
areas, evaluation, member, and any additional hierarchical 
levels. The productivity analysis report presents average 

35 scores for each equation of productivity data with 
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subtotals by member and any additional hierarchical levels. 
The quality question report presents the total score for 
each question and the percentage of maximum scores for each 
performance area. The productivity and quality comparison 
5 report presents average scores, maximum scores and 

percentage of maximum for both quality and productivity by 
member with comparison averages for a chosen hierarchical 
level. Additional detail and summary reports can be 
created by selecting various categories of information. 

10 The charts 46 display graphic and textual information 

useful in determining the overall performance of a member 
and identifying ways to improve that performance. In a 
particular embodiment, the charts 46 include sets of 
productivity and quality charts, productivity charts and 

15 quality charts. The productivity and quality charts plot 

productivity and quality scores for date, member, users and 
hierarchical levels. The productivity charts present 
productivity data plotted against time for members, users, 
and other hierarchical levels. The quality charts present 

20 quality data plotted against time for members, users, 

questions and hierarchical levels. 

The data importer 4 8 imports productivity data from 
external sources such as a telephony switch. The data 
importer 48 includes preprocessors 52 and a work area 54. 

25 The preprocessors 52 are tools used to create an 

instruction file for interpreting the information in a data 
file. The instruction files are operable to parse data 
files. In one embodiment, the preprocessors 52 include 
Nortel Datastream, Rockwell Spectrum IMS, and Monarch. The 

30 Nortel Datastream preprocessor is used for converting 

Nortel Datastream report information into a delimited 
format. The Rockwell Spectrum IMS preprocessor is used for 
converting Rockwell report t information. The Monarch 
preprocessor can be used to create custom preprocessors. 

35 Accordingly, the performance evaluation system 10 can be 
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configured to import any type of suitable data file 56 from 
an external device 58. 

The preprocessors 52 create a delimited file 60 from 
a data file 56. In the delimited file 60, columns of data 
are each separated by a delimiter character such as a 
comma, hyphen, colon, and the like. The delimiter acts as 
reference point telling the data importer 48 where one 
column of data stops and a new column starts. If the data 
file 56 is already in a delimited format, a preprocessor 52 
is not used. 

The delimited file 60 is a temporary file stored in 
the work area 54. As described in more detail below, the 
delimited file 60 is moved to the database space 16 and the 
productivity data mapped into the database based on the 
configuration defined in the database. Accordingly, the 
evaluation process is streamlined by having data in a 
central location for analysis. 

The database space 16 includes a database manager 70 
and a database 72. The database manager 70 calls stored 
procedures 80 to access the database 72. The stored 
procedures 80 are a precompiled collection of SQL 
statements and optional control-of-f low statements stored 
under a name and processed as a unit. The stored 
procedures 80 are stored within a database 72, can be 
executed with one call from the database manager 70, and 
allow user-declared variables, conditional executions, and 
other programming features. 

In addition to the stored procedures 80, the database 
72 includes organization tables 82, privileges tables 84, 
0 plans tables 86, data import tables 88, productivity tables 

90, evaluation tables 92 and filter tables 94. As 
described in more detail below, the organization tables 82 
allow an enterprise to set up the performance evaluation 
system 10 to correspond to its organizational structure. 
5 The privileges tables 84 store user access privileges based 
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on views of the organizational structure and on class of 
services. The plan tables 86 store questions, performance 
areas, and guidelines for generating performance 
evaluations in the performance evaluation system 10. The 
5 data import tables 88 store configurations for importing 

productivity data into the performance evaluation system 
10. The productivity tables 90 store productivity data for 
members of the performance evaluation system 10. The 
evaluation tables 92 store responses and scores for 

10 completed evaluations. The filter tables 94 store filters 

for sorting data and displaying results in the reports 44 
and charts 46. 

In a particular embodiment, the client and server 
platforms 18 and 20 are each a personal computer connected 

15 via the Internet. In this embodiment, the client and 

server 18 and 2 0 may each operate using MICROSOFT WINDOWS 
NT version 4.0 that provides TCP/IP connectivity over the 
Internet. The web-browser 3 0 may be Internet Explorer 3.2x 
or higher. The server engine 40 may be Internet 

20 Information Server 3.0 or higher. The database may be a 

relational database and the database manager 70 an SQL 
Server 6.5. 

The web-base architecture of the performance 
evaluation system 10 allows performance evaluation 

25 methodology to be standardized throughout an enterprise. 

In addition, users can take advantage of the Internet or 
other network to remotely access the performance evaluation 
system and complete member evaluations. Members can also 
remotely track their progress. 

30 FIGURE 2 illustrates an exemplary call center 100 

using the performance evaluation system 10. Although the 
performance evaluation system 10 will be described in 
connection with the exemplary call center 100, it will be 
understood that the performance evaluation system 10 may be 



Docket No. 019143.0272 



PATENT 



used for other types of enterprises, including other types 
of companies, businesses, groups, and organizations. 

Referring to FIGURE 2, the call center 100 includes a 
server 102 connected to a plurality of clients 104 through 
5 a local area network (LAN) 106. The call center 100 

includes a system administrator (user 1) , a service manager 
(user 10) , a technical manager (user 20) , a sales manager 
(user 30), a product A manager (user 31), and a product B 
manager (user 35) . Service agents (members 11 and 12) 

10 report to the service manager. Technical agents (members 

21 and 22) report to the technical manager. Product A 
agents (members 32 and 33) report to the product A manager. 
Product B agents (members 36 and 37) report to the product 
B manager. In addition, the product A and B managers 

15 report to the sales manager. 

FIGURE 3 illustrates details of the organization 
tables 82. The organization tables 82 allow an enterprise 
to set up the performance evaluation system 10 to fit the 
organizational structure of the enterprise. Thus, the 

20 organization tables 82 may store any number of user-defined 

levels in any user-defined hierarchy. In this way, quality 
and productivity tools are customized for the enterprise. 

Referring to FIGURE 3, the organization tables 80 
include a level table 120, an interlevel assignment table 

25 122, member tables 124, a user table 126, a user-member 

table 128, a keyword table 130, and a keyword assignment 
table 132. The level table 120 lists all the levels of the 
enterprise by a level ID. The levels are a collection of 
members who can be grouped together. The level IDs may be 

30 any suitable identifier operable to uniquely identify the 

levels. For the call center 100 of FIGURE 2, the levels 
are the company, the service group, the technical group, 
the sales group, the product A team, and the product B 
team. 
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The interlevel assignment table 122 assigns lower 
level in the enterprise hierarchy to a higher level in the 
hierarchy. In the interlevel assignment table 122, a first 
field identifies a higher level to which a lower level is 
5 assigned and a second field identifies the lower level. 

The labels "first" and "second" are used to distinguish 
between fields and not to specify a particular order for 
the fields. The lower levels may be assigned to a next 
higher level in the hierarchy or to each higher level in 

10 the hierarchy. The levels are identified in the interlevel 

assignment table 122 by the level IDs. For the call center 
100 of FIGURE 2, the service, technical, and sales group 
levels are assigned to the company level and the product A 
and B team levels are assigned to the sales group level. 

15 The member tables 124 list in a first field all the 

members of the enterprise by a member ID. The members are 
those individuals of the enterprise on which functions of 
the system 10 are performed. Each member ID may be unique 
codes assigned by the performance evaluation system 10 when 

20 the member is first defined in the system 10. 

In the member attribute tables 124, a second field 
assigns the member to a level. Additional fields provide 
member attributes, such as first and last name, password, 
comments, and the like. In another embodiment, some or all 

25 of the member attributes may be stored in a separate table. 

For the call center 100 of FIGURE 2, service agents 
(members 11 and 12) are assigned to the service group, 
technical agents (members 21 and 22) are assigned to the 
technical group, product A agents (members 31 and 32) are 

3 0 assigned to the product A team, and so on through product 

B agent (member 37) assigned to product A team. 

The user table 126 lists all the users of the 
enterprise by a user ID. The users are people in the 
performance evaluation system 10 that perform the 

3 5 evaluations and that carry out the various tasks associated 
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with the evaluation process such as defining all of the 
information needed to perform the evaluations. The user 
IDs may be any suitable identifier operable to uniquely 
identify the users. For the call center 100 of FIGURE 2, 
5 the users are the system administrator (user 1) , the 

service manager (user 10), the technical manager (user 20), 
the sales manager (user 30) , the product A manager (user 
31), and the product B manager (user 35). 

The users-member table 128 cross-references user IDs 

10 with member IDs for users who are also members in the 

system 10. A user is also a member when the user reports 
to and is evaluated by a user higher in the enterprise 
hierarchy. For the call center 100 of FIGURE 2, users 31 
and 35 report to user 30. Accordingly, users 31 and 35 are 

15 cross-referenced as members in the users-member table 128. 

The keyword table 130 lists all the keywords defined 
in the performance evaluation system 10 by a keyword ID. 
The keywords provide an easy way to classify members and 
other items of the performance evaluation system 10. The 

20 keywords can also be used to filter performance data for 

analysis or printing reports. The keyword IDs may be any 
suitable identifier operable to uniquely identify the 
keywords in the system 10. For the call center 100 of 
FIGURE 2, the keywords may relate to the experience of an 

25 agent or to an agency from which the agent was hired. In 

this way, the performance of the agents may be compared 
based on their experience, ~he agency from which they were 
hired, and the like. 

The keyword assignment table 132 assigns keywords to 

30 a member. In the keyword assignment table 132, a first 

field identifies a member by a member ID and a second field 
identifies a keyword assigned to the member by a keyword 
ID. One or more keywords may be assigned to each of the 
members. For the call center 100 of FIGURE 2, if the 

35 service agent (member 11) is hired from the AA Agency and 
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has less than two years' experience, keywords "Temp-AA 
Agency" and "2 years experience" are assigned to the 
service agent (member 11) . 

FIGURE 4 illustrates details of the privilege tables 
84. The privilege tables 84 assign each user a view and a 
class of services. The view specifies the levels and 
members of an organizational structure to which the user is 
allowed access. The class of services specifies services 
of the performance evaluation system 10 that the user is 
allowed to perform. The user has access privileges to 
perform services within the user's class of services for 
levels and members within the user's view. In this way, 
access privileges may be easily updated and maintained for 
the user in response to changes in allowed services or 
organizational structure. As a result, system 

administration cost is reduced. 

Referring to FIGURE 4, the privilege tables 84 include 
a user view table 140, a class of service assignment table 
142, and a class of service template table 144. The user 
view table 140 specifies levels and members of the 
organizational structure to which the user is allowed 
access . 

In the user view table 140, a first field identifies 
a user by user ID, a second field identifies a level or 
member associated with the user by the level or member ID, 
a third field identifies whether the item in the second 
field is a level or a member, and a fourth field enables 
the stored association between a user and a level or member 
in the user's view. Thus, in this embodiment, each user is 
associated with each level in the user's branch of the 
organizational structure and with members that report to 
the user. Each association between a user and a level or 
member may be enabled or disabled in the user's view and 
deselected using column 196 to be excluded from the user's 
view. In another embodiment, the user view table 140 may 
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include only associations that are included in the user's 
view. In this embodiment, the fourth field is omitted. 

For the call center 100 of FIGURE 2, the system 
administrator (user 1) is associated with each level and 
5 member in the performance evaluation system 10. All of the 

associations for the system administrator are enabled and 
thus included in the system administrator's view to allow 
the system administrator to access each level and member of 
the system. The service manager (user 10) is associated 

10 with each level in the service manager's branch of the 

organizational structure and with members that report to 
the service manager. Thus, the service manager is 
associated with the company and service group levels and 
with the service agents (members 10 and 11) that report to 

15 the service manager. The associations between the service 

manager, the service level and the service agents are 
enabled to allow the service manager to access the level 
and members to configure and perform performance 
evaluations and other functions for the members and level. 
0 The association with the company level is not enabled and 

thus excluded from the service manager's view. Similarly, 
the product B manager (user 35) is associated with the 
company, sales group, and product A and B team levels and 
with the product B agents (members 36 and 37) that report 
5 to the product B manager. The associations with the 

product B team level and product B agents are enabled and 
included in the product B manager's view. Access to the 
company and the sales group levels are not enabled and thus 
excluded from the product B manager's view. 
0 The user view table 140 may be manually populated by 

the system administrator or other user having suitable 
access privileges or may be generated based on the 
organizational structure defined by the organization tables 
82. In the latter embodiment, a user assignment table is 
5 used to assign each user to a level. Based on that 
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assignment, the organization tables 82 are interrogated to 
determine the levels within the user's branch of the 
organizational structure and the members that report to the 
user. This information is then used to populate the user 
5 view table 140. After the table is populated, associations 

are to be included in the user's view. 

The class of service assignment table 142 assigns a 
class of service template to a user. In the class of 
service assignment table 142, a first field identifies the 

10 user by the user ID and a second field identifies the class 

of service template by a service template ID. The service 
template ID may be any suitable identifier operable to 
uniquely identify the service templates. Use of the 
service templates allows a particular class of services to 

15 be defined and reused for any number of users. For 

example, for the call center 100 of FIGURE 2, group and 
team templates may be defined and associated with the group 
and team managers, respectively. Accordingly, the 

performance evaluation system 10 is more efficiently 

20 configured and administration cost is reduced. 

The class of service template table 144 specifies the 
class of services for each service template. The class of 
service template table 144 includes a first field 
identifying a service template by the service template ID. 

25 Additional fields are included to specify the services to 

be included in the service template. The class of service 
template table 154 also includes a field for identifying a 
specific user by the user ID. This is used to generate a 
custom class of services for the user rather than a service 

30 template. 

The services are predefined for the performance 
evaluation system 10. In a particular embodiment, the 
services may include administer productivity values, 
administer productivity data elements, access charting and 

35 reporting and administer filters, administer keywords, 



Docket No. 019143.0272 



PATENT 



administer member information, administer class of service, 
administer evaluation expressions, configure data import 
settings, import data, administer organizational structure, 
perform reviews, add or modify reviews, delete reviews, 
5 administer performance areas and questions, browse my 

reviews, browse my performed reviews, browse all reviews, 
and administer user information. 

FIGURE 5 illustrates details of the plan tables 86. 
The plan tables 8 6 store questions, performance areas 

10 constructed of a set of questions, and guidelines 

constructed of a set of performance areas. The questions, 
performance areas, and guidelines are used to generate 
performance evaluations for members of the performance 
evaluation system 10. The use and sharing of questions by 

15 multiple performance areas and the use and sharing of 

performance areas by multiple guidelines allows evaluations 
to be efficiently defined and automatically generated. In 
addition, members in a group are evaluated against the same 
set of criteria to provide fairness and objectivity. As a 

20 result, managers can perform consistent, objective 

evaluations efficiently. 

Referring to FIGURE 5, the plan tables include a 
question table 150, an answer types table 152, a 
performance area table 154, a question assignment table 

25 156, an expression table 158, an expression assignment 

table 160, a guideline table 162, and a performance area 
assignment table 164. The question table 150 defines 
questions that will be used in the performance evaluations. 
The question table 150 includes a first field identifying 

30 the question by a question ID, a second field providing a 

title for the question, a third field specifying text for 
the question, and a fourth field assigning a weight to the 
question. The question ID may be any suitable identifier 
operable to uniquely identify the questions. 
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For the call center 100 of FIGURE 2, a first question 
may be directed toward schedule adherence and ask "How 
often does the agent work on the originally assigned 
shift?" and a second question to be directed toward 
5 promptness and ask "How often is the agent late for work?". 

Relative weights are assigned to the question depending on 
their importance to the enterprise. Thus, if promptness is 
more important to the enterprise than schedule adherence, 
the second question will be assigned a greater weight. 

10 The answer types table 152 associates each question 

with a predefined answer type and attributes for that 
answer type. In the answer types table 152, a first field 
identifies the question by the question ID, a second field 
identifies the answer type by an answer type ID, and a 

15 third field identifies the attributes for the answer type. 

The answer type IDs may be any suitable identifier operable 
to uniquely identify the answer types. 

In a particular embodiment, defined answer types 
include yes/no, drop-down lists, horizontal scroll bar, 

20 edit box, and radio button answers. The yes/no answer type 

asks for either a yes or a no response. The drop-down list 
asks for one answer from a list of possible answers. The 
horizontal scroll bar answer type asks for an answer on a 
continuum for an overall rating. The edit box answer type 

25 asks for a numeric answer. The radio buttons ask for one 

answer from a list of possible answers. Attributes for the 
answer types include where appropriate, target values, 
labels, maximum and minimum values, and the like. 

The performance area table 154 lists all the 

30 performance areas in the performance evaluation system 10 

by a performance area ID. Each performance area is a 
grouping of questions that relates to a particular area of 
job performance. The performance area IDs may be any 
suitable identifier operable to uniquely identify the 

35 performance areas. For the call center 100 of FIGURE 2, 
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closing . 

The question assignment table 156 assigns questions to 
the performance areas. In the question assignment table 
5 156, a first field identifies a performance area by a 

performance area ID and a second field identifies a 
question assigned to the performance area by a question ID. 
One or more questions may be assigned to each of the 
performance areas. In addition, questions may be reused 

10 between performance areas. For example, performance areas 

2, 3 and 4 may each include question 3. 

The expression table 158 lists in a first field all 
the expressions in the performance evaluation system 10 by 
an expression ID. The expression IDs may be any suitable 

15 identifier operable to uniquely identify the expressions. 

A second field specifies a mathematical equation for the 
expression. The equation may contain data elements, 
numeric values, mathematical operations or mathematical 
functions. In a particular embodiment, the mathematical 

20 operations include addition, subtraction, multiplication 

and division. The mathematical functions include sum, 
average, minimum, maximum, and peak. 

The expression assignment table 160 assigns 
expressions to the performance areas. When associated with 

25 a performance area, the expressions are used to compute a 

productivity score for that performance area. In the 
expression assignment table 160, a first field identifies 
a performance area by a performance area ID and a second 
field identifies an expression assigned to the performance 

30 area by an expression ID. One or more expressions may be 

assigned to each of the performance areas. In addition, an 
expression may be reused between performance areas. For 
example, performance areas 1 and 6 may each include 
expression 1. 



Docket No. 



019143.0272 



PATENT 



20 

The guideline table 162 lists all the guidelines in 
the performance evaluation system 10 by a guideline ID. 
Each guideline combines one or more performance areas that 
are of interest for a particular evaluation. The guideline 
5 IDs may be any suitable identifier operable to uniquely 

identify the guidelines. For the call center 100 of 

FIGURE 2, the performance areas may include technical 
group, product A team, product B team, and the like. 

The performance area assignment table 164 assigns 

10 performance areas to the guidelines. In the performance 

area assignment table 164, a first field identifies a 
guideline by a guidelines ID and a second field identifies 
a performance area assigned to the guidelines by a 
performance area ID. One or more performance areas may be 

15 assigned to each of the guidelines. In addition, 

performance areas may be reused between guidelines. For 
the call center 100 of FIGURE 2, for example, the service, 
technical, product A team, and product B team guidelines 
may each include performance area 1. 

20 FIGURE 6 illustrates details of the productivity 

tables 90. The productivity tables 90 store productivity 
data for members of the performance evaluation system 10. 
The productivity data is used by the expressions in the 
performance areas to calculate productivity scores. 

25 Referring to FIGURE 6, the productivity tables 90 

include a data element table 166 and a productivity table 
168. The data element table 166 lists all the data 
elements in the performance evaluation system 10 by a data 
element ID. The data elements are items of information 

30 used to measure the productivity of members in the 

performance evaluation system 10. The data element IDs may 
be any suitable identifier operable to uniquely identify 
the data elements. 

The productivity table 168 includes a first field 

35 identifying members with a member ID, a second field 
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providing dates for the records, a third field identifying 
the data element for the record by a data element ID, and 
a fourth field storing the value of the data element. 

FIGURE 7 illustrates details of the data import tables 
5 88. The data import tables 88 import productivity data 

into the performance evaluation system 10 from external 
devices and automatically insert it into the production 
table. As a result, the evaluation process is streamlined 
by having data in a centralized location for data analysis. 

10 Referring to FIGURE 7, the data import tables 88 

include configuration tables 170, an import template 
assignment table 172, and an import template table 174. 
Each configuration table 170 provides information for 
identifying and mapping data from the delimited file 60 to 

15 the productivity table 90. 

The configuration table 170 includes a first field 
identifying a column in the delimited file 60 containing a 
data item corresponding to a data element, a second field 
identifying a data name, a third field identifying a data 

20 type, a fourth field identifying a data format, and a fifth 

field mapping the data item to the data element in the 
productivity table 90. In a particular embodiment, the 
data types include identity, date, duration and numeric 
types. An identity column provides identifying information 

25 such as a name or code. The date column provides the date 

of the information. The duration column shows an amount of 
time for the information. A numeric column contains a 
value . 

The import template assignment table 172 assigns an 
3 0 import template to a configuration. In the import template 

assignment table 172, a first field identifies the 
configuration by a configuration ID and a second field 
identifies the import template by an import template ID. 
The configuration and import template IDs may be any 
35 suitable identifiers operable to uniquely identify the 
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configurations and import templates, respectively. Use of 
the import templates allows a process for imported data to 
be defined and reused for any number of configurations. 
Accordingly, the performance evaluation system 10 is more 
5 efficiently configured and administration cost is reduced. 

The import template table 174 specifies preprocessor 
and other information for importing the data file 56. The 
import template table 174 includes a first field 
identifying the import templates by the import template 

10 IDs. A second field identifies a preprocessor, if any, to 

be applied to the data file 56 to generate the delimited 
file 60. A third field identifies the delimiter used in 
the delimited file 60. 

In operation, productivity data is imported by 

15 selecting a configuration, selecting users to determine how 

the data will be applied (members assigned to selected 
users receive the productivity data) , specifying how the 
date will be determined, selecting the data file 56 to 
import by selecting a drive and the file name, specifying 

20 how to match productivity data to members (by name or 

code) , and selecting whether or not to replace existing 
productivity information. In response to the import 
request, the data importer 48 retrieves the requested data 
file 56 and if the import template associated with the 

25 configuration requests a preprocessor 52, applies the 

preprocessor 52. Using the delimiter information in the 
import template for the configuration and the mapping 
information in the configuration table, the data importer 
48 inserts the productivity data into the productivity 

30 tables 90 for members in the user's view. 

FIGURE 8 illustrates details of an evaluation stored 
in the evaluation tables 92. As previously described, the 
evaluation is automatically generated based on guidelines, 
performance areas, and questions selected for the 

35 evaluation. 
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Referring to FIGURE 8, the evaluation includes a 
member ID 180 for whom the evaluation was completed, a date 
182 on which the evaluation was completed, and a plurality 
of performance areas 184. The performance areas 184 each 
5 include one or more questions 186 and their associated 

answer types 188. As the evaluation is completed, 
responses 190 are stored for each question and used to 
calculate a quality score 192 for the performance area 184. 
The quality score 192 is a measure of "goodness" for the 

10 member's performance. 

The quality score 192 for a performance area 184 is 
calculated based on the relative weight of the questions 
186 in the performance area 184. The score of responses 
190 to all questions 186 in the performance area 184 are 

15 summed and divided by the sum of the relative weights. 

This produces a quality score 192 for the performance area 
184. Mathematically, this is expressed in the following 
way: 

fff ac ^"^ )*relWeight ) 

V V target, - worst, ) ) 

£ relWeight i 

where 

20 n is the number of applicable questions in the 

performance area 

i ranges over the answered questions in the 
performance area 

achievedi is the number of points the member actually 
25 earned for question i 

worsts is the lowest possible score that can be 
assigned to question i 

relWeight 1 is the relative weight for question i 

30 After the quality scores 192 have been calculated for 

the performance areas 184 in an evaluation 92, the quality 
scores 192 are summed and divided by the sum of the 
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relative weight for all questions to derive a total 
evaluation quality score 194. Mathematically, this is 
expressed as: 

J] qScore x 
J2 relWeight x 



n is the number of applicable questions in the 
evaluation 

i ranges over the applicable questions in the 
evaluation 

10 qScorei is the q-score for question i 

relWeighti is the relative weight for question i 

The performance areas 184 may also each include one or 
more expressions 200 to calculate a productivity score 202. 

15 As previously described, the expression 200 is a user- 

defined formula to calculate a measure of productivity 
against a target score. 

The calculation of productivity score 202 for a 
performance area 184 is a direct computation of the 

20 expression 200 using the productivity data. After the 

productivity score has been calculated for all performance 
areas 184 that have an associated expression 200, the 
performance area productivity scores 202 are tallied to 
derive a total productivity score 204 for the evaluation. 

25 Mathematically, this is expressed as a median score of all 

performance area productivity scores: 
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where 

n is the number of performance areas in the evaluation 
having an associated evaluation expression that can be 
successfully calculated 
5 i ranges over the performance areas in the evaluation 

achievedi is the number of points the member actually 
earned for performance area i 

target^ is the target value for performance area i 

10 In another embodiment, the productivity scores for the 

different performance areas 220 may be weighted based on 
importance. In this embodiment, the overall productivity 
score for the evaluation is calculated by the following 
eguation: 

r-^ achieved l 



n is the number of performance areas in the evaluation 
having an associated evaluation expression that can be 
successfully calculated 

i ranges over the performance areas in the evaluation 
2 0 w, is the importance factor associated with performance 

i's p-score 

achieved., is the number of points the member actually 
earned for performance area i 

target, is the target value for performance area i 

25 

In operation, the user selects the member 180 and an 
existing guideline or one or more performance areas 184 for 
the evaluation. The evaluation is then automatically 
generated. Quality scores are generated based on the 

30 user's responses. Productivity scores are generated based 

on productivity data. 

FIGURE 9 illustrates details of the filter tables 94. 
The filter allows the user to define certain criteria for 
reporting and charting. This criteria is used to sort the 

35 data and display results in the charts and reports. 
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Referring to FIGURE 9, the filter tables 94 include a 
first field identifying the filters with a filter ID. The 
filter ID may be any suitable identifier operable to 
uniquely identify the filters. A second field indicates 
5 whether the filter includes plan criteria. Each category 

of plan criteria is specified by a separate look-up table. 
In a particular embodiment, the categories include 
evaluations, guidelines, performance areas, questions, 
expressions, and evaluators. In this embodiment, an 

10 evaluation filter table associates the filter with 

evaluations, a guideline filter table associates the filter 
with an identified guideline, a performance area filter 
table associates the filter with a specified performance 
area, a question filter table specifies questions for the 

15 filter, an expressions filter table specifies expressions 

for the filter and an evaluators filter table specifies 
evaluators for the filter. Each of these filter's tables 
include a first field identifying the filter with a filter 
ID and a second field identifying the relevant evaluation, 

20 guideline, performance area, question, expression or 

evaluator for the filter. 

Returning to the filter table 94, a set of date fields 
specify date(s) for the filter. The date may be a current 
day, current month, current quarter, current week, current 

25 year, previous day, previous month, previous quarter, 

previous week, previous year, or be otherwise defined by 
the user. A set of data fields specify data for the 
filter. The data may be members, all members, or member 
levels. A set of averages fields specify averages for the 

30 filter. The averages may be daily, weekly, bi-weekly, or 

monthly. A set of sample size fields specify sample size 
for the filter. The sample size allows data to be filtered 
based on any sample size entered by the user. A keyword 
field indicates whether the filter includes any keywords. 
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Keywords are associated with the filter in a separate look- 
up table as previously described for plan criteria. 

Thus, evaluators, guidelines, performance areas, 
questions, expressions, dates, data, averages, keywords, 
5 sample size, and other criteria may be specified for the 

filter. As a result, a user may narrowly define 
information and obtain meaningful results. 

Although the present invention has been described with 
several embodiments, various changes and modifications may 
10 be suggested to one skill in the art. It is intended that 

the present invention encompass such changes and 
modifications as fall within the scope of the appended 
claims . 
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WHAT I S/ CLAIMED IS: 

A method for importing a data file into a 
performance evaluation system, comprising: 

storing a plurality of user-defined data elements; and 
storing a user-defined configuration for the data 
file, the configuration operable to identify a data item in 
the data file and to map the data item to a data element. 



2. The method of Claim 1, the configuration further 
10 operable to associate the data item with a member of the 

performance evaluation system. 

3. The method of Claim 1, wherein the data file is 
a delimited file. 

15 

4. The method of Claim 1, if the data file is not a 
delimited file: 

storing a preprocessor file operable to generate a 
delimited file from the data file; and 
20 using the preprocessor file to generate the delimited 

file from the data file. 



6. The method of Claim 1, further comprising the 
configuration operable to identify a type for the data 
25 item. 



7. The method of Claim 1, further comprising the 
configuration operable to identify a format for the data 
item. 



30 
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8/. A performance evaluation system, comprising: 
/a first database table operable to store a plurality 
of user-defined data elements; and 

a second database table operable to store 
5 configuration information for importing a data file into 

the performance evaluation system, the configuration 
information operable to identify a data item in the data 
file and to map the data item to a data element. 



10 9. The performance evaluation system of Claim 8, 

further comprising a configuration including the 
configuration information and an identifier for associating 
the data item to a member of the performance evaluation 
system. 
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SYSTEM AND METHOD FOR IMPORTING PERFORMANCE DATA 
INTO A PERFORMANCE EVALUATION SYSTEM 
ABSTRACT OF THE DISCLOSURE 

A data file is imported into the performance 
evaluation system by storing a plurality of user-defined 
data elements. A user-defined configuration is stored for 
5 the data file. The configuration is operable to identify 

a data item in the data file and to map the data item to a 
data element in the performance evaluation system. 
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DECLARATION AND POWER OF ATTORNEY 
As a below named inventor, I declare that: 

My residence, post office address and citizenship are as 
stated below next to my name; that I believe I am the original, 
first and sole inventor (if only one name is listed below) or an 
original, first and joint inventor (if plural names are listed 
below) of the subject matter which is claimed and for which a 
patent is sought on the invention or design entitled SYSTEM AND 
METHOD FOR IMPORTING PERFORMANCE DATA INTO A PERFORMANCE 
EVALUATION SYSTEM the specification of which was filed July 1, 

1998, as Application Serial No. ; that I have reviewed 

and understand the contents of the above -identified 
specification, including the claims, as amended by any amendment 
referred to above; and that I acknowledge the duty to disclose 
to the U.S. Patent and Trademark Office all information known to 
me to be material to patentability as defined in 3 7 C.F.R. 
§ 1.56. 

I hereby claim foreign priority benefits under 3 5 U.S.C. 
§ 119 of any foreign application (s) for patent or inventor's 
certificate listed below and have also identified below any 
foreign application (s) for patent or inventor's certificate 
having a filing date before that of the application on which 
priority is claimed: 

Priority 
Date Claimed 
Number Country Filed (Yes) (No) 

NONE 

I hereby claim the benefit under 35 U.S.C. § 120 of any 
United States application (s) listed below and, insofar as the 
subject matter of each of the claims of this application is not 
disclosed in the prior United States application (s) in the manner 
provided by the first paragraph of 35 U.S.C. § 112, I acknowledge 
the duty to disclose to the U.S. Patent and Trademark Office all 
information known to me to be material to patentability as 
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defined in 37 C.F.R. § 1.56 which became available between the 
filing date of the prior application ( s) and the national or PCT 
international filing date of this application: 

Application 

Serial Number Date Filed Status 



-NONE- 



I hereby claim the benefit under Title 35, United States 
Code, § 119(e) of any United States provisional application (s) 
listed below. 

60/084.794 May 8, 1998 

(Application No.) (Filing Date) 



I hereby appoint : 



Jerry W. Mills 


Reg . 


No. 


23, 005 


Robert M. Chiaviello, Jr. 


Reg. 


No. 


32,461 


Ann C. Livingston 


Reg. 


No. 


32,479 


Thomas R. Felger 


Reg. 


No. 


28, 842 


Charles S. Fish 


Reg. 


No. 


35, 870 


Wei Wei Jeang 


Reg . 


No. 


33,305 


Kevin J. Meek 


Reg . 


No. 


33 , 738 


Rodger L. Tate 


Reg. 


No. 


27,399 


Scott F . Partridge 


Reg . 


No. 


28, 142 


James B . Arpin 


Reg . 


No. 


33,470 


James Remenick 


Reg. 


No. 


36, 902 


Jay B . Johnson 


Reg. 


No. 


38, 193 


David G. Wille 


Reg . 


No. 


38, 363 


Anthony E. Peterman 


Reg. 


No. 


38, 270 


Barton E. Showalter 


Reg. 


No. 


38, 302 


Terry J. Stalford 


Reg. 


No. 


39, 522 


Bradley P. Williams 


Reg. 


No. 


40,227 


Roger J. Fulghum 


Reg. 


No. 


39, 678 


Robert T. Holland 


Reg. 


No. 


40, 020 


Harold E. Meier 


Reg. 


No. 


22,428 


Thomas R. Nesbitt, Jr. 


Reg. 


No. 


22, 075 


James J. Maune 


Reg. 


NO. 


26, 946 



all of the firm of Baker & Botts, L.L.P., my attorneys with full 
power of substitution and revocation, to prosecute this 
application and to transact all business in the United States 
Patent and Trademark Office connected therewith, and to file and 
prosecute any international patent applications filed thereon 
before any international authorities. 
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Send Correspondence To : 
Terry J. Stalford 
Baker & Botts, L.L.P. 
2 0 01 Ross Avenue 
Dallas, TX 75201-2980 



Direct Telephone Calls To: 

Terry J. Stalford 

at (214) 953-6477 

Attorney Docket No. 019143.0269 



I declare that all statements made herein of my own knowledge are 
true and that all statements made on information and belief are 
believed to be true; and further that these statements were made 
with the knowledge that willful false statements and the like so 
made are punishable by fine or imprisonment, or both, under 
Section 1001 of Title 18 of the United States Code, and that such 
willful false statements may jeopardize the validity of the 
application or any patent issuing thereon. 



Full name of the first inventor 

Michael C. Powers 

Inventor's signature 

Date 

Residence (City, County, State) Irving, Dallas County, Texas 

Citizenship United States of America 

Post Office Address 812 W. Royal Lane #342 

Irving, Texas 75039 
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Full name of the second inventor 

Douglas A. Sudberry 

Inventor's signature 

Date 

Residence (City, County, State) Grapevine, Tarrant County, 

Texas 

Citizenship United States of America 

Post Office Address 3318 Summerfield Drive 

Grapevine, Texas 76051 
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Full name of the third inventor 

Inventor's signature 
Date 

Residence (City, County, State) 

Citizenship 

Post Office Address 



Irving, Dallas County, Texas 

United States of America 

2508 Syracuse 
Irving, Texas 75062 



DAL01: 387231 
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Full name of the fourth inventor 

Robert S. Bennett 

Inventor's signature 

Date 

Residence (City, County, State) Franklin, Williamson County, 

Tennessee 

Citizenship United States of America 

Post Office Address 3 02 Cypress Court 

Franklin, Tennessee 37069 



DAL01: 387231 
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Full name of the fifth inventor 

Inventor's signature 
Date 

Residence (City, County, State) 

Citizenship 

Post Office Address 



Clifford R . Phillips 



Bedford, Tarrant County, Texas 

United States of America 

3200 Misty Court 
Bedford, Texas 76021 



DA101: 337231 



